Bulk operations in MongoDB are not atomic across multiple documents, meaning individual write operations within the batch are atomic, but the batch as a whole is not
Bulk operations in MongoDB are a way to perform multiple write operations (like inserts, updates, or deletes) in a single database command, which can significantly improve network efficiency. However, contrary to a common misconception, a bulk operation is not an atomic transaction. If your bulk operation contains writes to multiple documents, it does not have the 'all-or-nothing' property. Some writes can succeed while others fail, and these partial results are committed to the database .
It's important to understand that MongoDB's atomicity guarantee still applies at the single-document level. This means that within a bulk operation, each individual insertOne, updateOne, or deleteOne operation is atomic for the document it affects . For example, if you have a bulk operation that updates Document A and then inserts Document B, the update to Document A is an atomic operation. If it succeeds, the change to Document A is persisted, even if the subsequent insertion of Document B fails .
Ordered Operations (Default): When you execute an ordered bulk operation, MongoDB processes the operations in the sequence you provided. If an error occurs during the processing of one of the write operations, MongoDB returns without processing any remaining operations in the list . In this case, the point of failure is clear, but all operations before the error are persisted.
Unordered Operations: With an unordered operation, MongoDB can execute the operations in parallel to maximize throughput. If an error occurs during the processing of a write operation, MongoDB will continue to process the remaining write operations in the list . The final result will contain information about which operations failed and which succeeded.
Key Takeaway: In both ordered and unordered modes, writes that succeed before an error are committed to the database and are not rolled back.
For use cases that require true atomicity across multiple documents (an 'all-or-nothing' guarantee), MongoDB has supported multi-document ACID transactions since version 4.0 . By wrapping your bulk operation logic within a transaction, you ensure that if any operation fails, the entire set of changes is rolled back. However, it's worth noting that transactions come with a performance overhead. The MongoDB documentation suggests that a large majority (80-90%) of applications using the document model can achieve data consistency without them by properly structuring their data and leveraging the atomicity of single-document operations . For your use case of scraping and upserting 1 million products, an unordered bulkWrite with upserts remains the most efficient approach, as the occasional failure of a single product upsert likely does not necessitate rolling back the entire batch.